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£\j ^ Abstract. Hidden regular variation defines a subfamily of distributions satisfying regular variation 

on E = [0, oo] d \{(0, 0, • • • , 0)} and models another regular variation on the sub-cone E' 2 ' = E\uf = i 
Li, where Li is the i-th axis. We extend the concept of hidden regular variation to sub-cones of 
^ | E' 2 ' as well. We suggest a procedure for detecting the presence of hidden regular variation, and if it 

exists, propose a method of estimating the limit measure exploiting its semi-parametric structure. 
We exhibit examples where hidden regular variation yields better estimates of probabilities of risk 
sets. 
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^ ■ 1. Introduction 

-i— > 

Multivariate risks with Pareto-like tails are usually modeled using the theory of regular variation 
on cones. Let C be a cone in [— oo,oo] a! satisfying x G C implies tx £ C for t > and denote the 
set of all non- negative Radon measures on C by M + (C). The distribution of a random vector Z is 
CN ■ regularly varying on C if there exist a scaling function g(t) f oo, and a non-zero Radon measure 

X G M + (C) such that 

Z 



> 
OO 

in 

o ■ (1-1) tP 
in 



9(t) 



x(-) 



in M_|_(C), where A denotes vague convergence [29]. Risks with heavy tails could also be modeled 
by stable distributions on a general convex cone; see [8]. 

Suppose the distribution of a random vector Z is regularly varying on the first quadrant E : = 
[0, oo] rf \{(0, 0, • • • 0)} as in (1.1) with limit measure v. It is possible for v to give zero mass to a 
proper sub-cone Cc E; for example, we could have 

^ i C = E( 2 ) = E\ „ , d {x jl = 0, • • • , = 0}, 

the first quadrant with the axes removed. If the distribution of Z is also regularly varying on the 
subcone C with scaling function gc(t) f oo and g{t)/gc{t) — > oo, then we say the distribution of 
Z possesses hidden regular variation (HRV) on C. HRV helps detect finer structure that may be 
ignored by regular variation on E . We will later refine our definition of hidden regular variation for 
a finite sequence of cones E D Ci D C2 D • • • C m . 

Failure of regular variation on E to distinguish between independence and asymptotic indepen- 
dence prodded Ledford and Tawn [19, 20] to define the coefficient of tail dependence and this idea 
was extended to hidden regular variation on E^ in [26]. See also [5, 9, 12, 15, 18, 21, 22, 23, 24, 
27, 32]. 

Hidden regular variation provides models that possess regular variation on E and asymptotic 
independence [28, pages 323-325]. The concept has typically been considered in two dimensions 
using the sub-cone E^ 2 ) . It is not clear how best to extend the ideas of HRV to dimensions higher 
than two and one obvious remark is that how one proceeds with definitions depends on the sort of 
risk regions being considered. 
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To demonstrate what is possible in higher dimensions, in this paper we define hidden regular 
variation on the sub-cones 

E« = [0, oo] d \ U^^...^^^ {x^ = 0, • • • , x^-^ = 0}, 3 < I < d, 

of E and show with an example that asymptotic independence is not a necessary condition for 
HRV on E^, 3 < I < d. Hidden regular variation on E®, means that the distribution of the 
random vector Z is regularly varying on E as in (1.1) with limit measure v and ^(E^" 1 )) > 0, but 
y(E") = 0. Also, there is a scaling function g E (i)(t) satisfying g(t)/ g E (i)(t) — > oo which makes the 
distribution of Z regularly varying on the cone E^ as in (1.1) with limit measure . Later, when 
we define HRV on the finite sequence of cones E D E*- 2 ** D • • • D E^ , our definition of HRV on E^ 
will be modified accordingly. We suggest exploratory methods for detecting the presence of hidden 
regular variation on E®, 2 < I < d. The existing method of detecting hidden regular variation 
on E^ 2 ) is valid only for dimension d = 2, but our detection methods are applicable for any finite 
dimension. 

If exploratory detection methods confirm data is consistent with the hypothesis of regular vari- 
ation on a cone E^ as in (1.1), we must estimate the limit measure . Previous methods [15] 
for estimating the limit measure z/ 2 -* of hidden regular variation on E^ 2 ) have been non-parametric 
and ignored the semi-parametric structure of u^ 2 \ We offer some improvement by exploiting the 
semi-parametric structure of ir& and estimate the parametric and non-parametric parts of 
separately. 

On E, estimation of the limit measure of regular variation is resolved by the familiar method of 
the polar coordinate transformation x — > (||x||,x/||x||); after this transformation, the limit measure 
v is a product of a probability measure S and a Pareto measure v a , v a ((r,oo\) = r~ a ,r > [28, 
pages 168-179]. Trying to decompose in this way presents the difficulty that the decomposition 
gives a Pareto measure v a (2) and a possibly infinite Radon measure [28, pages 324-339]. So we 
transform to a different coordinate system after which is a product of a Pareto measure v a (2) 
and a probability measure 

5(2) on £n(2) = { x g E< 2 > : xW = 1}, where x^ 2 ) is the second largest 
component of x. We call the probability measure the hidden angular measure on E< 2 ) . We 
suggest procedures for consistently estimating the parameter of the Pareto measure f Q (2) and 
the hidden angular measure S^ 2 ** and explain how these estimates lead to an estimate of . If HRV 
on E^ is present for some 3 < / < d, there is a similar transformation of coordinates making 
product of a Pareto measure v a (i) and a probability measure 

5(0 on $N(9 = {x G : xW = 1}, 
where is the Z-th largest component of x. We call this probability measure , the hidden 
angular measure on E^ and employ similar estimation methods for / > 3 as we did for / = 2. 

For empirical exploration of the spectral or hidden spectral measures, it is often desirable to 
make density plots. However, the hidden spectral measure is supported on <5^(w , which is a 
difficult plotting domain. For example, when d = 3, the set is a disjoint union of six rectangles 
lying on three different planes as shown in Figure 1. Though <$nW is a (d — l)-dimensional set, 
(i-dimensional vectors are needed to represent Stt^. So, the density plots on also requires an 
additional dimension. In the two dimensional case, the problem is resolved by taking a transfor- 
mation of points from = {x £ E : x^ = 1} to [0, 1] and looking at the density of the induced 
probability measure of the transformed points [28, pages 316-321]. We seek similar appropriate 
transformations in higher dimensional cases. We devise a transformation of points from <5H^) to 
the (d — l)-dimensional simplex A^_i = {x G [0, l] d_1 : Ylt=x x% — 1} ( see Section 4.1). The prob- 
ability measure jS® on the transformed points induced by is called the transformed (hidden) 
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Figure 1. The shaded region shows the set <5^( 2 ), when d = 3. 

spectral measure. Since the set Ad-i is represented by (d — l)-dimensional vectors, the problem of 
incorporating an additional dimension in the density plots vanishes. 

For characterizations of hidden regular variation [21] it is useful to know if i/^({x £ E^ : ||x|| > 
1}) is finite or not, where ||x|| is any norm of x. Such knowledge is also useful for estimating 
probabilities of some risk sets. For example, if z/W({x g E^ : ||x|| > 1}) is finite, then so is 

i/W({x e E^ : aix 1 + a 2 x 2 H h a d x d > y}, ai > 0, i = 1, 2, • • • ,d,y > 0. We show that this issue 

can be resolved by checking a moment condition. 

1.1. Outline. Section 1.2 explains notation. In Section 2, we review the definitions of regular 
variation on E and hidden regular variation on E*- 2 ), and extend the concept to the sub-cones 
E w = [0,oo] d \ Ui< jl<h< ... <jd _ l+1 < d {x jl = 0, ••• ,2^-'+! = 0}, 3 < I < d. Section 3 discusses 
exploratory detection techniques for hidden regular variation on E^ and estimation of the limit 
measure . We consider in Section 4 a transformation that allows us to visualize the hidden 
angular measure through another probability measure on the (d— l)-dimensional simplex. 
In Section 5, we discuss conditions for z/(^({x € E^ : ||x|| > 1}) being finite or not. Section 6 
gives examples of risk sets where hidden regular variation helps in obtaining finer estimates of their 
probabilities. Our methodologies are applied to two examples in Section 7. We conclude with some 
remarks and outline open issues in Section 8. 

1.2. Notation. 

1.2.1. Vectors and cones. For denoting a vector and its components, we use: 

x = (x l ,x 2 , • • • , x d ), x % = i-th component of x, i = 1, 2, • • • , d. 

The vectors of all zeros, all ones and all infinities are denoted by = (0, 0, • • • , 0), 1 = (1, 1, • • • , 1) 
and oo = (oo,oo,-- - ,oo) respectively. Operations on and between vectors are understood com- 
ponentwise. In particular, for non- negative vectors x and (3 = (/3 1 ,/? 2 ,--- ,/3 d ), write ~x? = 

((xT ,0T > ■ ■ >0 ) )• We denote the 

norm of x as ||x||. Unless specified, this could be 
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taken as any norm. For the i-th largest component of x, we use: 

= i-th largest component of x, i = 1, 2, • • • , d, i.e. > > • • • > x^ d \ 

So, the superscripts denote components of a vector and the ordered component is denoted by a 
parenthesis in the superscript. 

Sometimes, we have to sort the i-th largest components of the vectors Zx,Z2, ••• , Z n in non- 
increasing order. We first obtain the vector {Z^ \ • • • , Z„} by taking the i-th largest compo- 
nent for each Z,- and then sort these to get 

7 {i) ^ 7 {i) ^ 7 {i) 
Z (l) - Z (2) - " " " - Z (n)' 

We use the parentheses in the subscript to avoid double parentheses on the superscript. 
The cones we consider are 

E = E^ = [0, oo] d \{0} = [0, oo\ d \{x l = 0, • • • , x d = 0} 
= [0,oo] d \{xW = 0} = {x G [0,oo] d : x« > 0} 

and for 2 < I < d, 

E« =[0,oo] d \ U^^...^.^^ {x^ = 0, • • • ,x^+! = 0} 
= [0,oo] d \{x W = 0} = {x G [0,oo] d : x« > 0}. 

For 2 < I < d, E^ is the set of points in E such that at least I components are positive. Some- 
times E*- 2 ) is expressed as E*- 2 ) = E \ Lif =1 Lj, where Lj := {tei,t > 0} is the i-th axis and 
ej = (0, . . . , 0, 1, 0, . . . , 0), where 1 is in the i-th position, i = 1,2, ... ,d. For x G E, we use 
[0,x] c to mean [0,x] c = E\[0,x] = {y G E : V d =l y i /x i > 1}. 

1.2.2. Regular variation and vague convergence. We express vague convergence [28, page 173] 
of Radon measures as — > and weak convergence of probability measures [1, page 14] as =K Denote 
the set of non- negative Radon measures on a space F as M + (F) and the set of all non- negative 
continuous functions with compact support from F to R + as C^(F). The notation RV p means 
the family of one dimensional regularly varying functions with exponent of variation p ([28, page 
24], [2, 10]). For any measure m and a real- valued function /, denote the integral J /(x)m(dx) by 
m(f). 

For defining regular variation of distributions of random vectors on E = E^ 1 ) as in (1.1), we use 
the scaling function b(t) = b^\t) and get the limit measure v = Similarly, for defining regular 
variation of distributions of random vectors on E^\ 2 < / < d, we use the scaling function b^ l \t) 
and get the limit measure v">. For each 1 < I < d, define the set X® by = {x G E (z) : s(0 > 1}. 

is compact in E^, there always exists a suitable choice of the scaling function b^ l \t) 
which makes i/^(N^)) = 1. We assume this from now on. 

For each 2 < / < d, if we have hidden regular variation on E^, the limit measure can 
be expressed in a convenient coordinate system as a product of a Pareto measure v a (i){dr) = 
a^r~ a<l) ~ 1 dr, r > and a probability measure on the compact set = {x G E^ : = 

1}. The measure is called the hidden angular or hidden spectral measure on E^ . Whenever 
z^)({x G E^ : > l,a;W = oo}) = 0, we view through its transformed version denoted 
which is a probability measure on the (d — l)-dimensional simplex A^-i = {x G [0, 1] : 

Eti^<i}- 
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1.2.3. Anti-ranks. Suppose, Zi,Z2,--- , Z n are random vectors in [0,co) d . For j = 1,2, ... ,d, 
i = 1,2, ■ ■ ■ ,n, define the anti-rank 

J 



i=i 



for to be the number of j-th components greater than or equal to Zf . For 2 < I < d, define 

= the /-th largest component of = 1,2, ... ,d) 

and then order them as 



m (l) - m (2) - ^ m (ny 



2. Hidden regular variation 



We give more details about regular variation on E and HRV on E^ 2 ^ and then extend the defini- 
tions to hidden regular variation on sub-cones of E*- 2 ) . We illustrate with some examples. 

2.1. Hidden regular variation on E*- 2 -*. Consider regular variation on E and hidden regular 
variation on E^ 2 ^ . 

2.1.1. The standard case. The distribution of Z = {Z l ,Z 2 ,--- ,Z d ) is regularly varying on E := 
[0, oo] d \{0} with limit measure v if there exist a function bit) f oo as t — > oo and a non-negative 
non-degenerate Radon measure v ^ such that 



(2.1) tP 



Z 

W) G 



in M+(E). 



The limit measure v must have all non-zero marginals. Then, there exists a > such that &(•) G 
RVu a and v satisfies the scaling property 

(2.2) u(c-) = c~V(-), c > 0. 

Call the limit relation (2.1) the standard case which requires the same scaling function b{t) for all 
the components of Z in (2.1) and ensures that v has all non-zero marginals. 

HRV allows for another regular variation on a sub-cone such as K^ 2 \ The distribution of Z 
has hidden regular variation on E^ 2 ) if in addition to (2.1) there exist a non-decreasing function 
6 (2) (i) t oo such that b(t)/b^(t) -> oo and a non-negative Radon measure ' / on E (2) such 
that 

/ 2 )(-) inM + (E( 2 )); 



&( 2 )(t) 

see [28, page 324]. It follows from (2.3) that there exists > a such that b( 2 \-) G RV 1 / a (2) and 



(2.3) tP 

see [28, page 324]. It follows fro 
z^ 2 ** satisfies the scaling property 

(2.4) z/ 2 )( C .) = c-" ( V 2 )(.), OO. 

HRV implies z/(E^ 2 )) = 0, which is known as asymptotic independence [28, page 324]. We emphasize 
that the model of hidden regular variation on E^ 2 * 1 requires both (2.1) and (2.3) to be satisfied with 
b(t)/b( 2 \t) —7- oo, and not only regular variation on E^ 2 ) as in (2.3). 
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2.1.2. The non-standard case. Non-standard regular variation may hold when (2.1) fails, but 



(2.5) 



tP 



a? it) 



,3 = 1,2, 



d e 



A*0) mM + (E) 



for some scaling functions a 1 (-), a 2 (-), • • • ,a d (-) satisfying a l (t) f oo, where /i is a non-negative 
non-zero Radon measure on E [11, 30]. We assume that marginal convergences satisfy 



(2.6) 



tP 



aJ(t) 



e ■ 



i/^O) inM+((0,oo]), 



where v aj ((x , oo]) = x 133 , ft > 0,x > 0. Relation (2.5) is equivalent to 

'(j*~(Zi) 



(2.7) 



tP 



-J = 1,2,' 



d €• 



!/(■) inM + (E), 



where satisfies the scaling property z^(c-) = c 1 i y (-), c > 0, ([25, page 277], [10, 15]). The limit 
measures v and /i are related: 

(2.8) M[0,x] c ) = K[0,x^] c ), x € E . 

In this non-standard case, the distribution of Z has hidden regular variation on E*- 2 -* if, in addition to 
(2.7), there exist a non-decreasing function b^(t) f oo, such that t/b^(t) — > oo, and a non-negative 
non-zero Radon measure on E^ 2 ) satisfying 

(a^(^),j = l,2,... ,d) ^ 



(2.9) 



tP 



6( 2 ) (t) 



i/ 2 >(-) in M + (E< 2 »). 



Then, there exists c/ 2 ) > 1 such that b^ 2 \-) E RVi/ a ( 2 ) an d 1/1 satisfies the scaling property (2.4). 

Note that (2.7) standardizes (2.5) with scaling function b(t) = t, and the definition of hidden 
regular variation on E^ 2 ) in (2.9), is the most natural substitute for (2.3). This reduces the non- 
standard case to the standard one. Of course, we have to deal with the unknown nature of the 
scaling functions a? (•), j = 1, 2, ■ ■ ■ , d. 

2.2. Hidden regular variation beyond E^ 2 ). For dimension d > 2, it is possible to refine the 
model of HRV on E^ 2 ) by defining hidden regular variation on sub-cones of E*- 2 -* . For d > 2, even in 
the absence of asymptotic independence, it is possible to define HRV on sub-cones of E^ 2 ) and the 
family of distributions satisfying HRV on some sub-cone of E^ 2 ) is not a subfamily of distributions 
satisfying HRV on E^ 2 ). 

2.2.1. Motivation. A reason for seeking HRV on E^ is that in the presence of asymptotic indepen- 
dence when the limit measure v puts zero mass on E*- 2 **, regular variation on E may fail to provide 
non-zero estimates of the probabilities of remote critical sets such as failure regions (reliability), 
overflow regions (hydrology), and out-of-compliance regions (environmental protection). Beyond 
E< 2 >, if the limit measure in (2.3) puts zero mass on 

E (3) 

we would seek to refine HRV on E^' . 
Consider the following thought experiment. Suppose, Z = (Z 1 , Z 2 ,- ■ • , Z d ) represents concen- 
trations of a pollutant at d locations and that Z has a regularly varying distribution on E with 
asymptotic independence. Assume we found HRV on E^ 2 ) and the limiting measure in this 
case satisfies i/ 2 )(E (3) ) = 0, so HRV on E (2) , estimates P(Z^ > x 1; Z^ 2 > x 2 , • • • > x t ) to be 
for 3 < I < d and 1 < j\ < j 2 < ■ ■ ■ < ji < d. This resulting estimate seems crude and we seek a 
remedy by looking for finer structure of on the sub-cones E^ 3 ** D • • • D E^ in a sequential manner. 
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Another context for HRV on E® is as a refinement of regular variation on E when asymptotic 
independence is absent. Suppose, in the above thought experiment, Z has a regularly varying 
distribution on E with limit measure v such that z/(E (2) ) > 0, but z/(E (3) ) = 0. Asymptotic inde- 
pendence is absent, but P(Z^ 1 > xi, Z^ 2 > x 2 , • • • Z^ 1 > x{) is estimated to be for all 3 < / < d 
and 1 < ji < j2 < ■ ■ ■ < ji < d. This suggests seeking HRV on the sub-cones E^ D ■ ■ ■ D E^ . 

Examples in Section 2.3 show each modeling situation we considered in the above thought ex- 
periments can happen. 

We seek regular variation on the cones E D E^ 2 ) D E*- 3 ^ D • • • D E^ in a sequential manner. If 
for some 1 < j < <i, regular variation is present on E^\ as in (1.1) and the limit measure puts 
non-zero mass on E^, j < I < d, i.e. i/C?')(E^) > 0, then there is no need to seek HRV on any of 
the cones E^ J+1 ^ D ■ ■ ■ D E^. Recall the conventions that we replace v, a, E and b(t) by i/W, aW, 
E^ 1 ) and b^'(t) respectively. 

Of course, there are other ways to nest sub-regions of E and seek regular variation but our 
sequential search for regular variation on the cones is one structured approach to 

the problem of refined estimates. 

2.2.2. Formal definition of HRV on E^. The definition proceeds sequentially and begins with the 
standard case. Assume that Z satisfies regular variation on as in (2.1) and that we have regular 
variation on a sub-cone E^-* with scaling function b&\t) G an d limiting Radon measure 

i/tf) / 0. For j < I < d, further assume that i/tf)^ -1 )) > and i/W)(EW) = 0. The cone E^ 
could be E^ 1 ). The distribution of Z has hidden regular variation on E^, if in addition to regular 
variation on E (i) , there is a non-decreasing function b"'(t) f oo such that b^ (t)/b^\t) — > oo, and 
a non-negative Radon measure / on such that 



(2.10) 



tP 



b( l )(t) 



,(0f 



in M + (EW). 



From (2.10), there exists a^' > a^' such that G -R^i/q(0 an d has the scaling property 



(2.11) i/W(c-) = c- al V , )(-) I c>0. 

For vague convergence on E^, it is important to identify the compact sets of E^). From Propo- 
sition 6.1 of [28, page 171], the compact sets of E^ are closed sets contained in sets of the form 



{x G E (1) : x^ > wi,xi 2 > w 2 , 



wi,w 2 , ■ ■ ■ 



,wi > 0. So, for all 5 > 0, [x^ 1 ' > 5] is compact and 



> wi} for some 1 < j% < ji < • ■ ■ < ji < d and for some 



tP 



G {x G E 



(i) . x (l) 



>5} 



tP 



Z« bW(t). 



o. 



since bW(t)/b®(t) -> oo. Therefore, i/^(E (z) ) = is a necessary condition for HRV onE ( '. 

For defining HRV in the non-standard case, assume (2.7) holds on E^ 1 ^ and the rest of the defini- 
tion is the same with Z and 6 (1) (t) replaced by (a 1 ^(Z 1 ), a 2 ^(Z 2 ), • • • , a d ^ (Z d )) and t respectively. 

Remark 2.1. A few important remarks about hidden regular variation: 

(i) The definition of hidden regular variation leading to (2.10) is consistent with the definition of 
hidden regular variation on E^ 2 ) . 

(ii) The definition of regular variation on E^ 1 ) as in (2.1) or (2.7) requires that the limit measure 
i/W has non-zero marginals. When defining regular variation on E®, 2 < I < d, as in (2.10), 
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we do not demand such a condition. For instance, Z = (Z 1 , Z 2 , Z 3 ) being regularly varying 
on E^ 2 * 1 does not imply that (Z 1 , Z 2 ) is regularly varying on (0, oo] 2 . See Example 2.4. 
(iii) Non-standard regular variation allows each component Z 3 of the random vector Z to be scaled 
by a possibly different scaling function a 3 (t) as in (2.5). An alternative approach to defining 
regular variation on , 2 < I < d, would allow each component Z 3 of the random vector Z to 
be scaled by a possibly different scaling function &(" J '(t) and this would produce a more general 
model of HRV than the one we defined. However, we do not have a method of estimating 
the scaling functions b^' 3 (n/k); see estimation of b^ l \n/k) using (3.6) and estimation of 
a 3 (b^(n/k)) in the non-standard case using (6.9). 

2.3. Examples. We give examples to exhibit subtleties. Example 2.3 shows a model in which HRV 
is not present in E^ 2 ) but is present in E^ 3 ) . So, non-existence of HRV on E^ 2 ) does not preclude 
HRV on E^ 3 ) . Example 2.3 also shows that asymptotic independence is not a necessary condition 
for the presence of HRV on E^ 3 ) . In Examples 2.4 and 2.5, we learn that HRV on E^ 2 ) does not 
imply HRV on E® . In Example 2.5, HRV on E^ 3 ^ fails because i/( 2 \E^) > 0, but a different reason 
for failure holds in Example 2.4. In contrast, Example 2.2 demonstrates that HRV could be present 
on each of the sub-cones E^, 2 < I < d. Also, Example 2.5 shows that asymptotic independence, 
unlike independence, does not imply i/^(E^) = 0. 



Example 2.2. An extension of Example 5.1 of [21]: Suppose, Z 1 ,^ 2 , 
Then, regular variation of Z = (Z 1 , Z 2 , • • • , Z d ) is present on E with a - 
each of the sub-cones E^ with = I, for 2 < I < d. 



■ ■ , Z d are iid Pareto(l). 
1 and HRV is present on 



Example 2.3. Suppose, X and Y are iid Pareto(l) and Z = (X,2X,Y), so 

> u(-) in M_i_(E), 



IP 



Z 

2i € 



and v has all non-zero marginals. However, Z does not possess asymptotic independence since Z 1 
and Z are not asymptotically independent [25, page 296, Proposition 5.27] and thus HRV cannot 
be present on E*- 2 ) [28, page 325, Property 9.1]. However, 



,(E (3) ) = lim v({x. : x 1 A x 2 A x 3 > w}) 
= lim lim tP [X > tw,Y > tw] ■■ 

w— >0 1— >oo 



-- lim lim tP [X > tw, 2X > tw,Y > tw] 
w— >0 1— yco 

lim lim t^w)' 1 (tw)' 1 = 0. 



This suggests seeking HRV on E^ 3 ^ and indeed this holds with b^ 3 \t) = \Jt since for w\,W2,w-s > 0, 



lim tP 

t— >oo 



X > Viwi, 2X > Vtw 2 ,Y > Vi> 



W'3 



lim tP 

t— >oo 



x>Vt 



Wl v t) ' Y > ^ Ws 



lim t 

t— >oo 



Vi (w\ V 



«'2 



tw 3 ) 



"'3 



So, for this example, 

(i) Regular variation holds on E^ and E^ 2 ) (since v(E^) / 0), HRV holds on 
u(E^) = oo, v(E&) = 0. 

(ii) Asymptotic independence is absent but HRV on E 1 - 3 ^ is present. 

Example 2.4. Example 5.2 from [21]: Let, Xi,X 2 ,X 3 be iid Pareto(l) random variables. Also, 
let Bi,£>2 be iid Bernoulli random variables independent of (Xi,X 2 ,X 3 ) with P[Bi = 1] = P[Bi = 
0] = 1/2, i = 1,2. Define Z = (B 2 X U (1 - B 2 )X 2 , (1 - Bi)X 3 ). From [21], HRV exists on the cone 
E^ 2 ^ with q^ 2 ) = 2 and concentrates on [x 1 > 0, x 3 > 0] U [x 2 > 0,x 3 > 0]. Also, z/( 2 )({x : x l > 
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0,x 2 > 0}) = 0. Since, E< 3 > is a subset of {x : x 1 > 0,x 2 > 0}, u^(E^) = 0. However, HRV on 
E^ 3 ) fails. The compact sets of E^ 3 -* are contained in sets of the form {x : x 1 > w 1 , x 2 > u> 2 , x 3 > w 3 } 
for w 1 , w 2 , w 3 > 0. Since either Z 1 or Z 2 must be zero, for any increasing function h(t) f oo, and 1 
w l ,w 2 ,w 3 > 0, we have 



lim tP 

t— >oo 



h(t) 



£ {x : x 1 > w l ,x 2 > w 2 ,x 3 > w 3 } 



0. 



Hence, HRV holds on fi( 2) with b^(t) = y/i, but HRV on E< 3 > fails. 

Example 2.5. Let X\, X 2 and X 3 be iid Pareto(l) random variables and define Z = ((X1) 2 A 
(X 2 ) 2 , (X 2 ) 2 A (X 3 ) 2 , (Vi) 2 A (X 3 ) 2 ). First, note that 



tP 



Z 

3t £ 



in M + (E) 



for some non-zero Radon measure v on E with non-zero marginals. Also, 



tP 



t 2 /3 



for a non-zero Radon measure on E^ 2 * 1 . So, HRV exists on E^ 2 ) and hence, the components of 
Z are asymptotically independent [28, page 325, Property 9.1]. For wi, w 2 , w 3 > 0, 



lim tP 

t— >oo 



€ {x : x 1 > wi, x 2 > w 2 , x 3 > w 3 } 



lim tP 

t—too 



Xx > t l ' 3 { Wl \J w 3 ) l/2 ) X 2 > t 1/3 ( Wl V w 2 ) 1/2 ,X 3 > t 



1/3/ 



1/2 



3 ^ 1 ' (w 2 V w 3 ) 
z^^ 2 ^({x : x 1 > wi,x 2 > w 2 ,x 3 > w 3 }). 



yj (Wl V W 3 ) ■ (Wl V W 2 ) ■ (W 2 V W 3 

As {x : x 1 > wi,x 2 > w 2 ,x 3 > w 3 } C E^ 3 \ z/ 2 )(E^) > 0. So, for this example, 

(i) HRV exists on E^ 2 \ not on E^ 3 \ but Z is regularly varying on E^ in the sense of (1.1). 

(ii) Asymptotic independence holds but i/( 2 )(E^ 3 )) > 0. 

3. Exploratory detection and estimation techniques 

Existing exploratory detection techniques for HRV on E^ 2 ^ are valid in two dimensions. Our 
methods, applicable to any dimension, also allow for sequential search for HRV on E^\ 2 < I < d. 

We find a coordinate system in which the limit measure i/O in (2.10) is a product of a probability 
measure and a Pareto measure of the form u a (Q for some a"' > 0. Thus we exploit the semi- 
parametric nature of for estimation and detection. 

3.1. Decomposition of the limit measure v®. By a suitable choice of scaling function M^(t), 
we can and do make z^(N^) = 1, where = {x G E^ : > 1}. We decompose into 
a Pareto measure u m and a probability measure 

5(0 on = {x E : sW = 1} called the 

hidden spectral or hidden angular measure. 

Proposition 3.1. The distribution of the random vector Z has regular variation on EW, i.e. it 
satisfies (1.1) with C = E^ and % = v^ l \ and the condition z/( z )(nW) = 1 holds iff 

Z« Z \ 



(3.1) 



tP 



6(0(t)'Z(0 



i/ a( i)XS(')(.) inM + ((0,oo] x£N«), 
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where Z® is the Z-th largest component of Z. The limit measure v® and the probability measure 
S ® 

are related by 

(3.2) u®({* G E C0 : xW > r, G A}) = r^V^A), 

which holds for all r > and all Borel sets A C Sit®. 

Proof. See Appendix A. □ 

Remark 3.2. Proposition 3.1 only assumes regular variation on E^, a® > 0, whereas hidden 
regular variation on E® also requires (2.1) to hold and b(t)/b® (t) — > oo. 
Also, the convergence in (3.1) is equivalent to 

(i) Z® having regularly varying tail with index a® > and 

(ii) as t — > oo, 



P 



z® 



G • 



Z® > t 



S®{-) onSK®. 



R ^ A 

> r, G A 



t(t 1 /« (! V)" Q(0 5(0(A) =r -« (i, 5«(A). 



Remark 3.3. The polar coordinate transformation x i— >• (||x||,x/||x||) usually used for regular 
variation introduces a non-compact unit sphere {x G E^ : ||x|| = 1}. This defect is fixed by using 
Sit® instead. 

Example 3.4 uses Proposition 3.1 to construct random variables having regular variation on the 
cone E^ with the limit measure u®. 

Example 3.4. Suppose, (R, ©) is an independent pair of random variables on (0,oo] x Sit® with 

P[R >r ]= r" Q( °, r > 1, P[® G •] = S®(-). 

Then, 

By Proposition 3.1, the distribution of Z = R& is regularly varying on E® and satisfies (2.10) 
with v®(it®) = 1. This, however, does not guarantee regular variation on E. Also, unless has 
a support contained in {9 G Sit® : 6^ < oo}, the random variable Z might not be real-valued. 

3.2. Detection of HRV on E^ and estimation of v® . Is the model of hidden regular variation 
on E® appropriate for a given data set? If so, how do we estimate the limit measure u® and tail 
probabilities of the form P[Z n > z\, Z 12 > Z2, ■ ■ ■ Z H > z{\ for 1 < %\ < i% ■ • ■ < i\ < d. We consider 
the standard and non-standard cases and assume u® (it®) = 1. 

3.2.1. The standard case. Suppose, Zi, Z2, • • • , Z n are iid random vectors in [0, oo) d whose common 
distribution satisfies regular variation on E as in (2.1). We want to detect if HRV is present in E^ 
and this requires prior detection of regular variation on a bigger sub-cone E^' D E^ with the limit 
measure having the property u^(E®-^) > and u^(E®) = 0. Recall E^ could be E^. 

Here is a method for verifying that z/ J )(E(' -1 )) > and u^(E®) = 0. For each p > j, define 
a transformation : Sit^) ^ [0,1] as x ^ iW. If u^(E®-^) > and u0)(E®) = 0, then 

the probability measure o M® is degenerate at zero but o M®~V ' is not; see Remark 
4.2. As will be discussed later, we can construct an atomic measure , which consistently 
estimates S( j \ Using the atoms of o M®~^ , we plot a kernel density estimate of the density 
of o M®~^~ . If the plotted density appears to concentrate around zero, we believe that 
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j/C?)(E^ -1 ') = 0. Otherwise, we assume that ^(^(E^ -1 ') > 0. Then, using similar methods, we 
proceed to check whether i/( j ')(eO) = 0. 

Once convinced that z/j)(E (/_1) ) > and i^')(E (/) ) = 0, we seek HRV on E w . Using Proposition 
3.1, HRV implies 

(3.3) tP [z®/b®(t) e .] 4 ^ (0 (-) in M + ((0,oo]). 

So, we apply Hill, QQ and Pickands plots to the iid data {zf\i = 1,2, ••• , n} and attempt to 
infer that Z"' has a regularly varying distribution [28, Chapter 4]. 

If convinced that HRV is present, we estimate the limit measure . Define the set 

E lXoo = E« \ U 1 < jl<j2< ... <jl < d [x jl = oo, x» = oo, • • • , x> 1 = oo] = E« = oo] 
and the transformation : i— )■ (0, oo) x <5N^ as 

(3.4) Q«(x) = (x w ,x/x (/)N 



From (3.2) and the fact that is one-one, we get for any Borel set A C E^, 

uW(A) = (inE;^) = x ^^("(inE^)). 

So, estimating a® and the hidden spectral measure is equivalent to estimating i/W. 

We estimate a^' using one dimensional methods such as the Hill, QQ or Pickands estimator 
applied to the iid data {zf\i = 1,2, ••• ,n}. An estimator of can be constructed using 
standard ideas as follows [15]. Suppose, k(n) — > oo, k(n)/n — > 0, as n — > oo. Using Theorem 5.3(h) 
of [28, page 139], we get 

1 n 

(3 - 5) E £ (zf )/6(0(| ), Zi /^)) X 

i=l v y 

on M_|_((0, oo] x <5N®). Choosing ([1, oo] x •) as the set in (3.5), gives an estimator of S^ l \ but this 
estimator uses the unknown b^ l \n/k), which must be replaced by a statistic. 

Order the observations {zf\i = 1,2, ■•• , n} as z9X > z9X > ■■■ > z9\ which are order 
statistics from a sample drawn from a regularly varying distribution. Using (3.3) and Theorem 4.2 
of [28, page 81], we get 

z (i) 

(3 ' 6) 6(0(n/fc) ~* L 

Then (3.5) and (3.6) yield 

( 3 - 7 ) (^E e (^ /6W(n/fe)lZl/ x(0)^S/ &( ° (»/*)) =* >< ^ 

on M_|_((0, oo] x <5N^)) x (0, oo]. Applying the almost surely continuous map 

{y x S, x) H> v x 5([x,oo] x •) 
to (3.7), the continuous mapping theorem [1, page 21] gives 
1 n 

fcE^)^),^^^ (nA),oo] x •) 

i=l ^ ' 



12 A. MITRA AND S. I. RESNICK 



1 n 

(3-8) =lY. e (z^/z^,zM l) ) {[lM X ' ] ^ ^>ai,oc])S«(.) = 

i=l 



on <5N^. Thus, a consistent estimator for S^(-) is %z2i=i e ( z 0) /z (i) z . /z w\ (t 1 ' °°] x or 



(3.9) S«>(.):= fi 72 ""^;, ■■ . 

' (fc) 

3.2.2. T/ie non-standard case. Suppose, Zi, Z2, . . . Z n are iid random vectors in [0, oo) d such that 
their common distribution satisfies non-standard regular variation (2.7) on E. We seek HRV on 
E«. HRV is defined sequentially, so if HRV on exists, 

(i) either (2.7) holds, and (a l ^(Z l ), i = l,2, ■■■ ,d) is standard regularly varying on 

with limit measures z/C?),i/W and scaling functions 6 W for 1 < j < Z < d and 
i/d')^'- 1 )) > and i/«)(eW) = 0, 

(ii) or (2.7) holds, (2.10) holds with Z replaced by (a^Z*), i = 1, 2, ■ ■ ■ , d) , and i/(E^ -1 ^) > 
and i/(E®) = 0. 

In each case, (2.7) holds, (2.10) holds for (a^(Z 4 ), i = 1,2,- •• ,d) and £/6®(£) -> 00. 

Recall the definitions of antiranks {r|,i = 1,2, ■■■ ,n,j = 1,2, ••• , (i}, Z-th largest components 
of {l/rj,j = 1,2,... ,d} denoted for each i, and order statistics of {mf\i = 1,2,-- ,n}, 
denoted {m?\,p = 1, 2, - - - , n}. Here is a method to detect HRV on ECO in the non-standard case. 

Proposition 3.5. Assume that Zi, Z2, • • • Z n are iid random vectors from a distribution on [0, oo) d 
that satifies both regular variation on E and HRV on EW, so that (2.7) holds and (2.10) holds with Z 
replaced by (a j ^{Z j ),j = 1,2, • • • ,d). We assume that i/W = 1. Then, we have on M+(E (/) ), 

1 n 

(3-10) ^l- y e ( ,^^) onM+(E«). 

i=l v ' 

Proof. For Z = d, the statement is the same as Proposition 2 of [15], except that instead of defining 
HRV on E^ 2 ), we have assumed HRV on E^ . The proof of the case 2 < Z < d is similar to the case 
for I = d and is omitted. □ 

Remark 3.6. In the case, Z = 2 < d, the only improvement of Proposition 3.5, over Proposition 2 
of [15] is that here we assume v^ > (K( 2 )) = 1 instead of assuming v y 1 ({x e E( 2 ) : rf =1 xi > 1}) = 1. 
We claim that if HRV on E 1 - 2 ) is present, the assumption ^ 2 \"ii^) = 1 could always be achieved 
by a suitable choice of b^ 2 \t), but if d > 2, this may not be true for the assumption of u^ 2 \{x £ 
E^ 2 ) : A^ =1 x J > 1}) = 1, as claimed in [15]. See Example 2.2 for an illustration. 

Proposition 3.5 gives us a consistent estimator of v^ l \ without using the semi-parametric structure 
of z/O resulting from (2.11) and we now exploit this structure. In the non-standard case, decomposi- 
tion of i/W is achieved as in Proposition 3.1, only the role of Z is played by (a?*~(Zi),j = 1, 2, • • • , d) . 
The limit measure v"' of (2.10) is related to the hidden spectral measure S® through (3.2), which 
acts as the definition of the hidden spectral measure SW in the non-standard case. 



Proposition 3.7. The following two statements are equivalent: 



HIDDEN REGULAR VARIATION 13 

(1) The estimator of based on ranks is consistent as k(n) — > oo, k(n)/n — > 0, and n —> oo; 
i.e. 

n 

-tY.^ci iw (0 ^ =>" (0 onM+(E«). 

i=l v ' 

(2) The estimator of v a (i) x S^' based on ranks is consistent as k(n) — > oo, k(n)/n — > 0, and 
?i — >■ oo; i.e. 

1 n 

(3 ' 12) E e ( m f)/ m |0, ( (1/ ^ /m C0 !<,<,)) =* "o<0 >< on M + ((0,oo] x 

Proof. See Appendix B. □ 

Detection of hidden regular variation on E^', for some 2 < I < d, requires the prior conclusion 
that (a J< ~ (Z l ), i = 1, 2, • • • , d) is standard regularly varying on a bigger sub-cone E^- 1 D E^. Using 
the rank transform, we explore for regular variation on E and then move sequentially through the 
cones E D E (2) D • • • . We also need v^> to satisfy z/(j)(E^ _1) ) > and z/tO(E w ) = which is 
verified using the hidden spectral measure S^' . Finally, we verify regular variation on the cone 
E^ . From Proposition 3.5 and Proposition 3.7, HRV on E^ implies 

1 n 

(3-13) - £ e m (0/ m (0 => ^(0 ° n M +((0> °°D- 

We can use, for example, a Hill plot to determine whether (3.13) is true since consistency of the Hill 
estimator is only dependent on the consistency of the tail empirical measure and does not require 
the tail empirical measure to be constructed using iid data. ([31], [28, page 80]). This gives us an 
exploratory method for detecting hidden regular variation on E (/) in the non-standard case. 

To estimate the limit measure i/W , it is again sufficient to estimate and the hidden spectral 
measure S®. Estimate aS 1 ' using, say, the Hill estimator based on the rank-based data {m\ , i = 
1,2, ••• , n} [28, Chapter 4] and using Proposition 3.5 and Proposition 3.7, we get in M+(<JNW) 
that jE^x W/m C0 f ((1/r , )/m (0 5 ^^([Loo] x •) => ^ (i) ([l,oo])5(0(0 = <?«(.) or 

(3.14) S«(.) := 1 '1 1 ' m i Si ) ^ 

This gives a consistent estimator of S^(-). 

4. A DIFFERENT REPRESENTATION OF THE HIDDEN SPECTRAL MEASURE 

As discussed in the introduction, we map points of to the (d — l)-dimensional simplex 
A d _i = {x G [0, if' 1 : Ei=! ^ < 1}. The probability measure jS® on the transformed points 
induced by S** 1 ' is called the transformed (hidden) spectral measure. However, we must make the 
standing assumption that 

(4.1) !/( ! )({x6l (,) :x® > l,a?W =oo}) = 0, for all 2 < / < d, 

whenever exists, since otherwise the transformation is not one-one. Assumption (4.1) is not 
very strong and most examples satisfy this assumption. Nonetheless, this assumption is not always 
true, as illustrated by examples in Section 4.4. Recall the conventions that we replace u, a, S and 
S by z/«, a«, SW and S« respectively. 
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4.1. The transformation. First note that ^(^({x G : = oo}) = due to the scaling 
property of in (2.2) and the compactness of {x G E 1 - 1 -* : x^ 1 ^ > 1} in E*- 1 ^. So we may modify 

(4.1) to include 1 = 1. 

For each I, 1 < I < d, define a transformation T® : i-> A d ^ =: {s G [0, if" 1 : X^=i s * ^ x }' 
which is one-one on an appropriate subset of 6^ l \ The appropriate subset is 

(4.2) L>P = {x G <5«W : < oo}. 
On D { 1\ define as 

(4.3) r (l) (x)- ( ^ 3 rf j) 

To identify T^(D^), first we define a map 0^ : A^_i — >• [0, 1] as 

d-l 

(4.4) </> (Z) (s\ s 2 , • • • , s d_1 ) = the Z-th largest component of (1 - ^ s\ s 1 , s 2 , ■ ■ ■ , s^ 1 ). 

i=l 

Using this notation, we see that 

(4.5) Df := T®(D®) = {(s\ s\ ■ ■ ■ , s^ 1 ) G A d _ x : <f>W(s\ s 2 , ■■■ , s^ 1 ) > 0} C A^-i- 
To show that is one-one on D± , we explicitly define the map as 



(4.6) TVr\s\s 2 ,...,s d -i) 

S , • • • , S a L / 

We extend our definition of from D^p to the entire set by setting T^(x) = for x G . 
We define a similar extension of 1 to the whole simplex A^-i by setting (s 1 , s 2 , ■ ■ ■ , s d_1 ) = 

1 for (s 1 , s 2 , • • ■ , s^ -1 ) G D2 • Now define the probability measure = o T® on A^-i; 
this is called the transformed hidden angular measure on EW. Note that, 

S«(Df )C ) = i/9({x G E (0 . x (0 > 1 * g L>? )C }) = i/«({x G E« : x» > l,x« = 00}) = 0. 

Therefore, using (4.5), we get S^{D^) = 1. Since is one-one on and S^(D)p) = 1, for 
any Borel set A C we can compute S^(A) by noting that 

(4.7) S (l) {A) = S^(AnDf ] ) = 5 (,) (rW(Anz>{ )). 

So, studying the transformed hidden angular measure S® on the nice set A^-i is sufficient to 
understand the hidden angular measure 

SCO. 

4.2. Estimation of S^. In the standard case, we get from (3.9), 

1 n 

(4.8) S(%) := ^ E V^))^ 00 ] X 

on M_|_((5H0). The function defined (4.3) is continuous on D^p and hence is continuous almost 
surely with respect to the probability measure ■ Therefore, by the continuous mapping theorem 
[1, page 21], 

1 n 

(4.9) ^oT(0-(0:=-£ e(z ^ 
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on M + (A d _i). Conversely, (4.9) implies (4.8) by continuity of T^" 1 on D$p and the fact S^(D^) = 
1. Thus (4.8) and (4.9) are equivalent. 

In the non-standard case, (3.14) implies that on M + (<^W), 

n 

By a similar argument as in the standard case, this is equivalent to the fact that on M+(A d _x), 
WoI®-\) := fcE^/^O,^^^.), ([Loo] x •) 5» or©" (.) = *«(.). 

4.3. Supports of transformed (hidden) spectral measure S^ 1 '. The following lemma illus- 
trates that the supports of the transformed (hidden) spectral measures are disjoint. 

Lemma 4.1. Recall defined in (4.5). For 1 < j < I < d, 

!/(?') (EW) = iff S®(D$) = 0. 

Proof. By the scaling property (2.2) or (2.11), u^\{x G E : sc^') = oo}) = 0, and hence, by the 
continuous mapping theorem, 

i/(j)(E(0) = i/0')(eW n{x G E : < oo}) = i/ aW x S®(QW(E® n{x G E : < oo})), 
where Q (j) ( x ) = ^jy) • Now , 

i/ a(j) x S®(Q®(E® n{x G E : a£') < oo})) = i/ a0) x S^({(r,0) G (0, oo) x SX® : W > 0}) 

= lim \- aU) S ij) {{0 G <$N^ : 9® > 0}) 

Hence, i/0')(eW) = iff S^)({0 G <JN&'> : (/) > 0}) = 0. Since S^(D[ j) ) = 1, where D[ j) is as 
given in (4.2), we get 

S®({0 G tfN<7) : (/) > 0}) = S^({0 G : fl(Q > 0} n D^) 

= SW(T®({0 G <W (j) : (/) > 0} n Z^)) 

= S^({( S \s 2 , ■■■ , 3 d - 1 ) G A d _! : 0«( S \ s 2 , • • • , Z" 1 ) > 0}) 
= S^(D^). 

Hence, the result follows. □ 

Remark 4.2. The fact that i/C?)(eW) = iff S<-?)({0 G <5K^ : 00) > 0}) = 0, follows from the 
proof of Lemma 4.1. Notice, this result does not require the assumption (4.1). 

If = and HRV on E w exists, then the support of is contained in and the 

support of is contained in d{ , which are disjoint. So, if one seeks (hidden) regular variation 
on the nested cones E = E (1) D E (2) D ••• D E^, if HRV is present, the transformed spectral 
measure and the transformed hidden spectral measures on A d _i will have disjoint supports. 

For a visual illustration, fix d = 3 and suppose is concentrated on the corner points of the 
triangle A2. By Lemma 4.1, z/( 1 )(E^ 2 )) = and we search for HRV on E^ 2 ). Assume that it is 
indeed present and so consider S^ 2 \ As we have already noticed, the support of is contained in 
and hence does not put any mass on the corner points of the triangle A2. Therefore, and 
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iSK 1 ) have disjoint supports. Two cases might arise from this situation. In the first case, puts 
positive mass in the interior of the triangle A2. Applying Lemma 4.1, we infer that > 

which rules out the possibility of HRV on E^ 3 ) . Hence, we do not consider . In the second case, 
is concentrated on the axes of the triangle A2 and by Lemma 4.1, v( 2 \E^) = 0. Hence, as 
usual, we search for HRV on E^ and let us assume that it is present. Then, we consider . As 
noted, the support of is contained in and hence it only puts mass in the interior of the 
triangle A2. Hence, in this case, all three of and have disjoint supports. 

Now, consider another case, where is not concentrated on the corner points of the triangle 
A2, but is concentrated on its axes. Using Lemma 4.1, z/(E^) > 0, but ^(E^) = 0. bo, we 
should not search for HRV on E^ and hence should not consider S^ 2 \ However, we consider 
presence of HRV on E^ 3 ) and hence consider . But, the support of is contained in the 
interior of the triangle A2 and hence does not put any mass on the axes. So, in this case also, 
we would consider only and which have disjoint supports. 

In the final case, suppose puts mass in the interior of the triangle A2. Lemma 4.1 implies 
i/WQEC 3 )) > and we should not seek HRV on any of the sub-cones E*- 2 -* or E^ 3 \ 

In all these illustrative cases, the transformed spectral measure and the transformed hidden 
spectral measures have disjoint supports. 

4.4. Lines through 00. Section 4 made the standing assumption (4.1), which is not always true. 
In Example 4.3, the measure concentrates on the lines through 00; i.e., on the set {x G E^ 2 ) : 
X W = 00}. Examples 4.4 and 4.5 show that for 2 < j < I < d, i/W({x G E (i) : x® > l,x (1) = 
00}) = does not imply u^\{x G E^ : > 1, a^ 1 ) = 00}) = and vice versa. 

Example 4.3. Let X and Y be two iid Pareto(l) random variables. Let B be another random 
variable independent of (X, Y) such that P[B = 0] = P[B = 1] = |. Define 

Z = (Z 1 , Z 2 ) = B(X, X 2 ) + (1 - B)(Y 2 , Y), 

so that 

tP [Z/t 2 G •] 4 in M + (E), 

where for w, v > 0, u((w,oo] x [0,oo]) = ^w" 1 / 2 , u([0, 00] x (v,oo\) = \v~ x l 2 and z/(E( 2 )) = 0. For 
w, v > 0, 



lim tP 

t— >oo 



— G (w, 00] x (v, 00] 



lim -P \X > tw, X 2 > tv] + lim -P \Y 2 > tw.Y > tv] 



= lim -P \X > tw] + lim -P \Y > tv] = - ( — + - 

So HRV exists on the cone E^ with limit measure such that 

i/ (2) (0,oo] x (u,ooj) = - ( - + - 

I \W V 

Hence, letting v — > 00, we get v^ 2 \{w, 00] x {00}) = ^ and similarly, i^ 2 )({oo} x (u,oo]) = i. So, 
we conclude that in this case, z/ 2 )({x G E^ : x*- 2 -* > 1, x^ = 00}) = 1. 

Example 4.4. Let X\, X 2 , ■ ■ ■ ,X§ be five iid Pareto(l) random variables. Let (B\, B 2 , -B3) be 
another set of random variables independent of (X\, X2, • • • , X$) such that P[Bi = 1] = 1 — P[B{ = 
0] = § and Yh=i B i = 1 - Now > defin e Z as 

Z = (Z\Z 2 ,Z S ) = B 1 (X 1 , X 2 ,0) + B 2 (X 2 ,X 2 ,0) + B 3 (X 2 ,X 2 ,X 2 ). 
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It follows that 



tP 



25t 2 /9 



inM+(E), 



where for w,v,x > 0, v((w,oo] x [0,oo] x [0,oo]) = \w 1//2 , i/([0, oo] x (v,oo] x [0,oo]) = |u 1 / 2 , 
i/([0,oo] x [0,oo] x (x,oo]) = \x~ 1/2 and z/(E (2) ) = 0. Now, we look for HRV on E (2) . Notice that 



tP 



5t/3 



G • 



4^(0 inM+(E( 2 )), 



where for w,v,x > 0, i/( 2 )((*uj, oo] x (y, oo] x [o,oo]) = if^ + ^ + H -1 ' 2 ), ^ {2) ( 

[0, oo] x 

(v,oo] x (x,oo]) = ±(ra)~ 1/2 , z/( 2 )((w,oo] x [0, oo] x (x,oo]) = \{xw)~ 1/2 and ^ 2 '(E( 3 ') = 0. Hence, 
letting v — > oo, we get 



z/ 2) ((u;,oo] x {oo} x [0, oo]) 



1 

5w ' 



and so v ( 2 )({x G E( 2 ) : x( 2 ) > 1, x 

(!) = oo}) > 0. We now seek HRV on the cone E (3) . For 

w,v,x > 0, 



lim tP 

t-^oo 



(t/3) 2/3 



G (w,oo] x (v, oo] x (x, oo] 



lim —P 

t— ►oo 3 



X 2 > (t/3) 2/3 w,X 2 > (t/3) 2/3 v,X 2 > (t/3) 2/3 x\ = (wvxy 1/2 . 

So, HRV exists on the cone E^ 3 ) with limit measure such that for w,v,x > 0, 

u^((w, oo] x (v, oo] x (x, oo]) = (wvx)" 1 / 2 . 

Hence, for this example, z/( 3 )({x G E^ 3 ^ : x^ 3 ) > l^x^ = oo}) = and thus for 2 < j < I < d, 
i/W({x G E (/) : > l,xW = oo}) = does not imply i/0')({ x G E (j) : x^ > l,xW = oo}) = 0. 

Example 4.5. Let Xi, X2, ■ ■ ■ ,X§ be five iid Pareto(l) random variables. Let (Bi, B2, B3) be 
another set of random variables independent of (Xi, X2, ■ ■ ■ , X5) such that P\B{ = 1] = 1 — P[Bi = 
0] = 3 and X^i=i B{ = 1. Now, define Z as 

Z = (Z\Z 2 ,Z 3 ) = B^Xi,*! 3 ,*?/ 4 ) + B 2 (V 3 ,V 2 ,X 2 5/4 ) + B 3 (X 3 , X 3 ,X 3 ). 



It follows that 



tP 



125t 3 /27 



i/(0 in M+(E), 



where for all w,v,x > 0, v((w, 00] x [0, 00] x [0, 00]) = jw 1 ' 3 , i/([0, 00] x (i>, 00] x [0, 00]) = %v 1 ' 3 , 
u([0, 00] x [0,oo] x (x, 00]) = 4x -1 / 3 and u(E^) = 0. Now, when we seek HRV on E^ 2 \ we get 



tP 



t 3 / 2 



G • 



4^(0 inAf + (E( 2 )), 



where for w,v,x > 0, ^ 2 \(w, 00] x («, 00] x [0,oo]) = i(w)" 1/3 , z/^( [0,oo] x (f, 00] x (x, 00]) = 
i(ra)" 1/3 , vW((w,oo] x [0,oo] x (x,oo]) = i(iw)" 1/3 and i/( 2 )(E^) = 0. Notice, i/ 2 )({x G E< 2 > : 
x^ 2 ) > 1, xW = 00}) = 0. Now, we look for HRV on the cone E^ 3 \ For w,v,x > 0, 



lim tP 

t— >oo 



— G (w, 00] x («, 00] x (x, 00] 
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lim — P 

t—t-oo 3 



Xi > tw, Xf > tv, X^ /A > tx 



+ lim -P 

t— >oo 3 



Xf > tw, X 2 > tv, Xo /4 > tx 



+ Urn -P [Xf > tw, Xf > tv, Xf > tx] 
lim -P [Xi > tw] + lim -P [X 2 > tv] 

t—>oo 3 t— >oo 3 

t 



+ lim -P 

t— >oo 3 



X 3 > {tw) 1/z ,X A > (tv) 1/3 ,X 5 > (tx) 1/3 
^ (w' 1 + w" 1 + (wvx)~ 1/3 ^j . 



So, HRV exists on the cone E^ 3 ) with limit measure such that 

u^((w, oo] x (v, oo] x (x, oo]) = - (w~ x + u" 1 + (ui-ux)^ 3 ^ . 

Following Example 4.3, i/ 3 )({x G E (3) : x (3) > = oo}) = 2/3 so that for 2 < j < I < d, 

i/G) ({x G E0) : xC?) > = oo}) = does not imply i/(0({ x G E^ : a;(0 > 1,# = oo}) = 0. 

5. Deciding finiteness of u^({x g E^) : ||x|| > 1}) 

For characterizations of HRV [21], it is useful to characterize when i/(')({x G E^) : ||x|| > 1}) 
is finite and when it is not, where ||x|| is any norm of x. Such characterizations are also useful 
for estimating risk set probabilities. For example, the limit measure i/W puts a finite mass on 
a risk set of the form {x G E^ : a\x x + a 2 x 2 + • • • + adX d > y}, at > Q,i = 1, 2, • • • ,d,y > 0, 
iff i/')({x G E^) : ||x|| > 1}) is finite. The HRV theory is not useful for estimation of risk set 
probability if the limit measure puts infinite mass on that risk region. 

The following section resolves this issue using a moment condition. Subsequently we show that 
for 2 < j < I < d, neither i/W({x G E (/) : ||x|| > 1}) being finite implies u^({x G E (j) : ||x|| > 1}) 
is finite, nor the reverse is true. 

5.1. A moment condition. The following theorem gives a necessary and sufficient condition for 
the finiteness of i/0({x G E^ : ||x|| > 1}). For d = 2, the condition of Theorem 5.1 is given in 
Proposition 5.1 of [21]. 

Theorem 5.1. For each I, 2 < I < d, the limit measure puts finite mass on the set {x G E^) : 
||x|| > 1}, i.e. i/W({ x e e(0 : ||x|| > 1}) is finite iff 



(5.1) 

Proof. We have, 



y(0 



< oo. 



Hence, the result follows. 



/W({x G EW : ||x|| > 1}) = i/W({ x e E^ : xW||JL.|| > 1}) 
= u aW x S®({(r,0) G (0,oo] x <W« : r||0|| > 1}) 

v ail) ({re(0,o O ]:r>l/\\0\\})S( l \de)= [ \\ 



AD 



□ 
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The following corollaries translate the condition of Theorem 5.1 to the transformed hidden an- 
gular measure 

Corollary 5.2. If i/W({x G E® : > l.acW = oo}) > 0, then i/W({x G E^ : ||x|| > 1}) is 
infinite. 

Proof. Observe, if we denote the largest component of 6 as we get 

S W ({6> G : 6>« = oo}) = i/ W ({x G E« : x® > 1, x (1) = oo}) > . 
Hence, the result follows from Theorem 5.1. □ 

Corollary 5.3. Suppose, i/W({x G E^ : s(9 > l,a?W = oo}) = 0. Then, i/W({x G E^ : ||x|| > 1}) 
is finite iff 

where cjP^ and are defined in (4.4) and (4.5) respectively. 

Proof. The condition S®(D^ C ) = i/W({x G E^ : > = oo}) = 0, where £>P is defined 

in (4.2), allows us to apply the change of variable formula to (5.1) using the almost surely one-one 
transformation as in (4.3). Now, the result follows from Theorem 5.1. □ 

Choosing the Li-norm in (5.2), we get the simple condition: z/')({x G E^ : ||x|| > 1}) < oo iff 

2 

5.2. A particular construction. We defined HRV on a series of sub-cones 
• • • D E^, and discussed the finiteness condition in Theorem 5.1 for each of the limit measures 
vQ\ 2 < I < d. A natural question is if for some 2 < j < I < d, HRV exists on both the cones E^) 
and E®, does finiteness of ^({x G E^ : ||x|| > 1}) imply finiteness of z/^({x G E^ : ||x|| > 1}) 
or vice versa? We construct an example to show that there are no such implications. 

Example 5.4. Suppose, Xj, i = 1, 2, • • • ,d are iid Pareto(l). Also, assume arc 
mutually independent random variables with Ri having distribution Paretof 'l^V ) • Now, for each 



2i+l 

2 < I < d, define a set of mutually independent random variables Sj, i = 2, 3, ■ ■ ■ , d, such that Sj 
has a distribution on {x G D 2 '■ x l = x l+1 = ■ ■ ■ = x^" 1 = 0}, where D 2 is defined in (4.5). 
Also, assume that (JQ, i = 1, 2, • • • ,d), (Ri,i = 2, 3, • • • ,d) and (sj, i = 2, 3, ■ ■ ■ , d) are independent 
of each other. Note that even though we have restricted the supports of the probability measures 
S^', we still have the flexibility to choose them in a way so that (5.2) is satisfied or not, depending 
on whether we want to make i/W({x G E^ : ||x|| > 1}) finite or infinite. 

Now, let (B\, £>2, ■ ' ' ,B d ) be another set of random variables independent of all the previous 
random variables such that P\Bi = 1] = 1 — P[Bi = 0] = 4 and Yli=i Bi = 1- Recall the definition 
of the transformation from (4.6), which maps points from of to 5^ = {x G E( z) : x® = 1}. 

Note that, the range of 1 is D± \ where is defined in (4.2). Now, define the random vector 
Z as 

Z = (Z\Z 2 ,--- ,Z d ) 

= B 1 (X 1 ,X 2 , ■ ■ ■ ,X d ) + B 2 R 2 TW~ 1 {s 2 ) + B 3 R 3 T®~\s 3 ) + • • • + B d R d T^"\s d ). 
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Since the range of T(0 1 is Df \ all the components of (s^) are finite, 2 < I < d, and hence 

all the components of Z are [0, oo)-valued. Also, 

tP[Z/t e ■] A inM+(E), 

where v([0, oo] x • • • x [0, oo] x (u, oo] x [0, oo] x • • • x [0, oo]) = (d • u)" 1 , where (u, oo] is in the i-th 
position and this holds for all 1 < % < d. Also, z/(E( 2 )) = 0. Notice, for each 2 < I < d, the parameter 
of the distribution of Ri is chosen in such a way that HRV of (X±,X2, ■ ■ ■ ,Xd) on E^0 or regular 
variation of R p T^~ (s p ) on E®, I < p < d, does not affect the HRV of Z on E®. Also, by choosing 



the support of S^ p \ 2 < p < d, to be concentrated on {x E D 



(p) . 



x 



p+i 



0} we 



have ensured that RpT^' (s p ), 2 < p < /, would not have any HRV on the cone EW. So, the only 
part of Z contributing in HRV on E (z) is #jT(9 (s { ), and therefore, for 2 < I < d and x > 0, 



lim tP 

t— yoo 



(t/d) 



(2Z+1)/Z(Z+1) 



lim — P 



> x, 



Pz 



Z(0 



lim — P 

t-Kx> a 



P, 



(t/d) 



(2Z+1)/Z(Z+1) 



> X 



p 



r (z)- 



lim — 

t-»oo a 



(t/d) 



(2i+l)/Z(Z+l), 



-Z(Z+1)/(2Z+1) 



P 



{t/d) (2i+iyi(i+i) 
(sz) e ■ 

T (Z) 



> x 



T (Z)" 



SZ € 



(sz) e • 



-Z(Z+1)/(2Z+1) 



P 



T (Z)" 



sz e 



Hence, following Proposition 3.1, for 2 < I < d, Z has regular variation on the cone E w with 



scaling function b^(t) = (t/d) 



(2Z+l)/Z(Z+l) o (q 



5(0 (.) = p r« (s,) e 



1(1 + 1)/ (2Z + 1) and hidden spectral measure 
Also, notice l/a(0 = + ppjl is a decreasing function in /, which 

indeed confirms that for 2 < j < I < d, b {j) (t)/b {l \t) ->■ oo, which is a required condition for HRV 
on E(0. So, Z has HRV on the each of the cones E(0 with the limit measure 2 < I < d. Now, 
we look for the transformed hidden spectral measure for the limit measure and show that 
it indeed coincides with 5(0. 

Since the hidden spectral measure 5(0 has been defined through the function T(0 which has 



range we have S^(D\ ) = 1, where D[ , where D[ is defined in (4.2). So, we get the 

transformed hidden spectral measure S^(-) as S^(-) = P[s/ E •]. So, this hidden transformed 
spectral measure matches with our earlier S®. Following the comments made before about 
5(0 , we have the flexibility to choose 5(0 in such a way that (5.2) is satisfied or not, and this could 
be done independently for each 2 < I < d. So, this example shows that we could construct a random 
variable which has regular variation on each of the cones E(0 with limit measure i/(0 2 < I < d, 
and for each 2 < / < d, we could independently choose to make z/(0({x E E(0 : ||x|| > 1}) finite 
or infinite. Therefore, for 2 < j < I < d, neither u^\{x. E E^-* : ||x|| > 1}) is finite implies 
z^(0({x E fi(0 : ||x|| > 1}) is finite , nor the reverse is true. 



6. Computation of probabilities of risk sets 



In this section, we consider two risk regions and illustrate how HRV helps obtain more accurate 
estimates of probabilities of risk sets. 
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6.1. At least one risk is large. One scenario has Z = (Z l ,Z 2 ,--- ,Z) representing risks such 
as pollutant concentrations at d sites [15]. A critical risk level, such as pollutant concentration t % 
(i = 1,2, ••• , d) at the i-th site, is set by a government agency. Exceeding t l for some i results 
in a fine and the event non-compliance is represented by Llf =l [Z l > t 1 ]. The probability of non- 
compliance is, 



P [non-compliance] = P[U? =1 {Z* > f}] = ^ P[Z l > **] - P[Z n > t h , Z i2 > f 2 ] 

i l<ii<i2<d 

+ --- + (-i) (i_1) P[Z h > t h ,Z i2 > ? 2 ,--- ,Z l i > 



+ ■■■ + (-l)^ PiZ 1 >t\Z 2 > t 2 , • • • ,Z d >t d }. 

Suppose, Z,Zi,Z2,-- - , Z„ are iid random vectors whose common distribution, for simplicity, is 
assumed standard regularly varying on E = E^ 1 -* with scaling function b(t) = b^(t) as in (2.1). 
Assume HRV holds on each of the cones E^ with scaling function b^(t) as in (2.10), 2 < I < d. 
Since asymptotic independence is present, relying only on regular variation on E means all the 
interaction terms in the inclusion-exclusion formula are estimated to be but HRV improves on 
this. 

Estimating P[Z l > 1 < i < d, is a standard procedure, perhaps using peaks over threshold 
and maximum likelihood; see [10, page 141], [4]. For 2 < j < d, 1 < i\ < i% < ■ ■ ■ ij < d, large k 
and large n/k, the probability P[Z H > f 1 , ■ ■ ■ , Z l i > t lj ] is approximated using HRV on E^ by 



P[Z h > f 1 ,--- ,Z** > fj] = P 



Z %1 t n Z l i t l i 

> TTTw TTTt , T77w 7TS > 



b(J)(n/k) Ui){n/ky ' bd)(n/k) b^){n/k) 



(6.1) « -u® fix G E^ : x 11 > • • • , > 



n 



6(j')( n /fc)' bV)(n/k) 



We need to estimate and b^\n/k). Notice that, for w 1 , ■ ■ ■ ,w° > 0, 
f{x G E^ : x h > w 1 , • • • ,x*s > 

= v aU) x S {j) (Ur,0) G (0,oo] x : r 0U > u, 1 ,.-- , r <9^ > 
Using (3.6), we get 

(6.3) zgj/ftW (ra/fc) 4 1, 

and thus we use Z^j as an estimator of b^ (n/k). From (6.1), (6.2) and (6.3), we approximate 
P[Z h >*«,••• > t*'] as 



-« 



0) 



V P=1 ""(ft) 

where and are the consistent estimates of and obtained in Section 3.2.1. 
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6.2. Linear combination of risks. A second kind of risk set used in hydrology [3, 9] is of the 

form {x G E : 71X 1 + 72X 2 + • • • + r Yd,x d > y} f° r 7« > 0, z = 1,2,--- , d and y > 0. Here the risks 
could be wind speed and wave height and a linear combination represents dike exceedance. Assume 
for simplicity d = 2 and note 

P\ lx Z X + l2 Z 2 >y]= P[ lx Z x > y] + P[ l2 Z 2 > y] - P[ lx Z x > y, l2 Z 2 > y] 

(6.4) + P[ ll Z x + l2 Z 2 >y, ll Z 1 <y, l2 Z 2 <y\. 

Suppose, Z, Zi,Z2, ••• , Z n are iid vectors whose common distribution has non-standard regular 
variation on E = E (1) as in (2.7) and HRV on E (2) with scaling function b^(t) as in (2.9). Asymp- 
totic independence holds and thus regular variation on E estimates the last two terms on the right 
hand side of (6.4) as zero. This is crude and HRV should improve the risk estimate. 

As in the previous scenario, estimating P[^iZ l > y], i = 1,2, using (2.6) is standard and we 
proceed to estimate P[^\Z X > y,^/ 2 Z 2 > y\. From Section 2.3 in [15], we have (2.9) equivalent to 



(6.5) tP 



Zi 

-.J =1,2 G 



4*< 2 >(0 inM + (E( 2 )), 



y(&( 2 )(0) 

where z^ 2 ) and are related by 

(6.6) i/( 2) ((x,oo]) = ^((x^oo]), xEE< 2 > 

where (3 = ((3 X ,(3 2 ) and ft, j = 1,2, is the marginal index of regular variation defined in (2.6). 
Using (6.5) and (6.6), we approximate P[y\Z x > y,^f 2 Z 2 > y] as 

P\ lx Z x > y, l2 Z 2 >y]=P \ . , Z ' . > , „,.,f 2 , 



a?-(bV)(n/k)) 7 ia 1 (6 (2) (^A))' a 2 (ti 2 )(n/k)) l2 a 2 (60') (n / 'k)) 

« /7x € E( 2 ) ■ x 1 > g x 2 > V W 

n l v \ Xfc ^ - X > 7ia i(6(2)( n /A;))' > 72 a 2 (&0)(n/£;)) 

(6.7) = f (x e E( 2 ) : x 1 > f —I- — V 1 ^ 2 > f 




^ 2 a 2 (b^(n/k)) 

We require estimates of v^ 2 \f3 % and a 1 (b^ 2 \n / k)) , i = 1,2. There are standard methods for 
estimating one dimensional indices = 1,2, based on (2.6) ([28, Chapter 4], [4, 10]) which yield 
consistent estimators (3 l ,i = 1,2. For v^ 2 \ observe, 

,( 2 )({xeE( 2 ):i 1 > 1 i ) 1 ,. 2 >w 2 }) 

= y a{ 2) x S (2) ({(r,0) £ (0,oo] x 5^ 2 ) : rfl 1 > it; 1 ,^ 2 > w 2 }) 

(6.8) .^(^V^^'^'W. (»W>0). 

Also, from Section 4.3 in [15], we get 
(6-9) Z\ m /a?(bW(n/kj)$l, 

where Z 3 , 2) is the [l/rnVJ-th largest order statistic of the j'-th components of Zj, i = 
1,2, •• • ,n. So, we use Z 3 (2) as an estimator of a 3 {b^> (n/k)) , j = 1,2. Finally, using (6.7), 

(rv m (fc)"i) 
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(6.8) and (6.9), we approximate P[~f\Z l > y,^f 2 Z 2 > y] as 
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(6.10) 



P\lxZ 1 >y, l2 Z 2 >y\^- [ 



-A( 2 ) 



V 



P=1 0P lp z p m 



S^(d9), 



where and are consistent estimates of and obtained in Section 3.2.2. 
Estimation of the fourth term of the right side of (6.4) requires care. First, observe 



PhiZ 1 + l2 Z 2 > y,^iZ l < y, l2 Z 2 < y] 



P 



Z 2 



a 2 (U 2 )(n/k)) y 



Z 2 



-z>^({x G E : 7 ia 1 (& (2) (n/A;))x 1 + l2 a 2 {b {2) {n/k))x 2 > y, 
7 ia 1 (^(?i/A:))x 1 < y, 7 2 a 2 (b (2) (n/fc))s 2 < y}) 



(6.11) 



-z>( 2 )({x G E : 7^1 (2) a; 1 + 72 Z 2 



n 



x 2 > y,jiZ) r 



^V722 2 n ,_( a)l ,X 2 < J/}). 



In the last approximation in (6.11), a J (b^ 2 > (n/k)) is replaced by Z 3 f2 , , j = 1,2, using (6.9). 

(ri/"V)D 

For 4>i,4>2 > 0, the set {x G E : (^ix 1 + </>2X 2 > y, ^ix 1 < y, (j) 2 x 2 < y} C E^ 2 - 1 is not a compact 
subset of E^ 2 \ so, p( 2 )({x G E : fax 1 + <j) 2 x 2 > y, fax 1 < y, 4> 2 x 2 < y}) could be infinite in which 
case it is not clear how HRV can refine the estimate of P\^\Z X + ^Z 2 > y,^\Z l < y,^ 2 Z 2 < y]. 
So, we must check finiteness of the quantity on the right side of (6.11). 

Set fa = jjZ 3 ^ (2) ii = 1)2 and define A := {x G E : fax 1 + 4> 2 x 2 > y,fax x < y,fax 2 < y}. 

Using (6.6) and following similar methods as in (6.2), we get 



P( 2 )({x G E -.fax 1 + fax 2 > y.fax 1 < y,<p 2 x 2 < y}) = / /3 1 /3 s V)^ L ~ 1) '(x 2 )^ >v {2) '(dx) 



[ (3 l p 2 {9 l )^- l \9 2 )^- 1 ^ 



lo2 



^1^1-1)^2)^-1) 



rW 1+ P 2 -Vu am (dr)S®(dO) 



(6.12) 



/3 1 + /3 2 - «( 2 ) - 2 

x (/3i +/ 32_ a (2)_ 2) 



ii^ 1 + cj) 2 9 2 



L^ 1 V c/> 2 fl 2 



(/3 1 +/3 2 -a< 2 )-2)' 



S^(d0). 



Finiteness of the quantity on the right hand side of (6.11) is equivalent to the finiteness of the 
quantity on the right hand side of (6.12) which is difficult to verify; see [15]. This problem is 
inherent in estimation for this type of risk region. 
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We proceed assuming the finiteness of z/( 2 )({x G E : (f)ix l + fax 2 > Uifax 1 < y,fax 2 < y}). 
From (6.11) and (6.12), we get for large k and n/k, the estimate, 



P^+^Z 2 > y^Z 1 < y, l2 Z 2 < j/] « - 



§. £ (gl)(/3 1 -l)(g2N(/3 2 -l) 
S»W /3 1 + /3 2 - d( 2 ) - 2 

( / §1 +/ 32_ ( J(2)_ 2) - 



(3l + ^2_ d (2)_ 2) 



^i^ 1 v fad 2 



S {2 \dO), 



where and are consistent estimates of offl and S^ 2 ) obtained in Section 3.2.2. 



7. Computational Examples 

This section considers the performance of the estimation procedure described in Section 6 on 
two data sets, one simulated and one consisting of Internet measurements. We also compare 
performance with Heffernan and Resnick [15]. 



7.1. Simulated data. We simulated iid samples {(Xi,Yi),i = 1, 2, ••• , n = 5000.}, where X\ 
Pareto(l), Y\ ~ Pareto(2) and X\ and Y\ are independent. Therefore, using (2.9) we get 

(7.1) v®((x,y),oo] = — , (x,y>0), 

xy 

and = 2 and i/( 2 )({x G E^ 2 ) : x^ > l,x^) = oo}) = 0. Using (4.7), we obtain the transformed 
hidden spectral measure 

5(2) 

(7.2) S^(-) = ^ ({x e E( 2 ) : x^ > 1, g j) . 

The density with respect to Lebesgue measure of £( 2 ) is 

f 1(1 - s)" 2 , if < s < |, 

(7.3) /(*) = < is" 2 , ifi<s<l, 

[ otherwise. 

We test accuracy of our estimates of and £K 2 )(-). The Hill plot for {m\ 2 \i = 1,2, •• • ,ra}, 
the plot of the estimated transformed hidden spectral densities for k = 500, 1000, and the plot 
of the actual transformed hidden spectral density (7.3) are shown in Figure 2. We also estimate 
probabilities of risk sets of the form P\X\ > t\,Y\ > t 2 ] for large thresholds t\ and t 2 . Using a 
method similar to the one used to obtain (6.10), we estimate the probability P\X\ > t±, Y\ > t 2 ] as 

F[X, > t ll Y l > H ~ h -[ ([!(*!_)''] \/\l(*-f]f m sV m , 

where Xm > Xm\ > ■ • • > -X"( n ) an d ^(l) — ^(2) > ' " ' > Yr n \ are the order statistics for {X^,i = 
1, • • • , n} and {Yi,i = 1, • • • , n}, and the remaining notation has the same meaning as in (6.10). 
Since we simulate the data we take f3\ = 1 and fa = 2 and concentrate on estimating using 
the Hill estimator and estimating by the formula given in (3.14). We compute the estimates of 
P[X\ > 100, Y"i > \/l0] for different values of k using these estimators and plot the graph in Figure 
3. The range of A; is A; = 500 to k = 5000. 
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Figure 2. Hill plot of and estimated and actual transformed hidden spectral densities 



A different estimate of P[X\ > ti,Y\ > £2] is given by Heffernan and Resnick [15]: 



P[X X >h,Y 1 >t 2 } 



k 



n 



,(2) 



x 



t 2 



Y, 



(n/"»[?)i) 



00 



where i>( 2 ) is defined in (3.11). We again use P\ = 1, $2 = 2 and estimate using the Hill 
estimator. Then, using the above estimator, we compute the probability P[X\ > 100, Y\ > \/l0] 
for different values of k and plot it as a graph in Figure 3. The values of k are chosen between 
k = 500 and k = 5000. 

Using the true distribution of (X 1} Yi), we calculate P[Xi > 100, Y x > y/W] = 0.001. In Figure 3, 
we observe that the plot of the risk estimates obtained using the Heffernan-Resnick [15] estimator 
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Figure 3. Plots of estimates of P[Xi > 100, Y\ > VTO] for different values of k 
(sample size = 5000) using both our and H-R(Heffernan-Resnick [15]) estimator 
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Figure 4. Plots of estimates of P[X\ > 100, Y"i > \/l0] for different values of k 
(sample size = 500) using both our and H-R(Heffernan-Resnick [15]) estimator 



is more stable but our current estimator of P[X\ > 100, Y\ > \/l0] is more accurate for most k in 
the range k = 500 to k = 5000. 

The Heffernan-Resnick [15] estimator of P[X\ > ti,Y\ > uses an empirical distribution 
function and thus is subject to the defect that a zero estimate is reported for the risk probability 
when t\ and ti are high but actually P\X\ > t±,Yi > t?\ is non-zero. Irrespective of how high the 
threshold is, our estimator does not estimate P\X\ > t±, Y\ > as zero, unless it is actually zero. 

As an illustration, we reduced the sample size to n = 500 and applied the two estimators of 
the risk probability P[X\ > 100, Yi > \/T0]. As suspected, the Heffernan-Resnick [15] estimator 
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estimates the probability P[X\ > 100, Y\ > VlO] as zero, whereas our estimator is still reasonably 
accurate. This is shown in Figure 4 where k ranges between k = 50 and k = 500. 

7.2. Internet traffic data. We analyze HTTP Internet response data consisting of sizes and 
durations of responses collected during a four hour period from 1-5 pm on April 26, 2001 by the 
University of North Carolina at Chapel Hill Department of Computer Science's Distributed and 
Real-Time Systems Group under the direction of Don Smith and Kevin Jaffey. This dataset was also 
analyzed in [15]. We investigate joint behavior of two variables - size of response and throughput 
(size of response/time duration of response) and estimate the probability that both the size and 
rate are big as a measure of burstiness. 

We start by estimating marginal tail parameters. We use QQ plots [28, page 97] (not shown 
here) to choose the value k = 5000 for both the variables size and rate. Using this k, we get the 
estimates of tail indices /3 1 = 1.15 and p 2 = 1.51 for size and rate using the QQ estimator. 

Next, we investigate presence of asymptotic independence by plotting an estimated density of the 
transformed spectral measure S(-), defined in Section 4.1. In agreement with Heffernan and Resnick 
[15], our estimated density plots for different values of k for the transformed spectral measure show 
two modes at the points and 1 and take values close to zero in between, thus indicating asymptotic 
independence of size and rate (plots are not shown). 

(2) 

Is hidden regular variation present? The Hill plot in Figure 5 of {m- , 1 < i < n} suggests this 
is so and we proceed to estimate the density of the transformed hidden spectral measure. Figure 5 
gives plots of the estimated transformed hidden spectral densities for k = 500, 1000, 5000. 

Next, we estimate probabilities of risk sets of the form [Size > x, Rate > y] which we consider as 
measures of burstiness. Examination of the (Size, Rate) data, indicates x = 2 x 10 7 and y = 10 5 arc 
reasonably high thresholds. We use both our estimator and the estimator given in [15] to compute 
P[Size > x, Rate > y] for different values of k from k = 500 to k = n and plot them in Figure 6. 

We also estimated P[Size > x,Rate > y] for higher thresholds x = 2 x 10 s and y = 10 7 , as a 
measure of extreme traffic burstiness. Again, we use both our estimator and the Heffernan-Resnick 
[15] estimator to estimate P[Size > x, Rate > y] for different values of k from k = 500 to k = n 
and plot them in Figure 7. If hidden regular variation is present for the pair (Size, Rate), then the 
actual risk probability cannot be zero. The Heffernan-Resnick [15] estimator reports an estimate 
of zero but ours does not. 

8. Concluding remarks 

Hidden regular variation provides a sub-family of the distributions having regular variation on 
E that is sometimes equipped to obtain more precise estimates of probabilities of certain risk sets, 
which are crudely estimated as zero by regular variation on E; two examples are shown in Section 
6. 

The theory of HRV has deficiencies. Consider d = 3, and on the planes of K^ 2 \ suppose the 
random vector Z has regular variation with three different tail indices a*- 2 )' 1 < a^' 2 < a^' 3 . As 
a convention, say Z has regular variation with tail index «( 2 )'* on {x 6 E^ 2 ** : x l = 0}, i = 1,2,3. 
If we follow our HRV model and method of estimation, we ignore the regular variation Z exhibits 
on {x G E^ 2 ) : x 2 = 0} with tail index a^' 2 , which is actually more important than the regular 
variation on E^ 3 * 1 . A way to repair this defect is the following alternative method: In d dimensions, 
first consider the big cone E, then consider all the (2) pairs of components of Z and their regular 
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Figure 5. Hill plot of and estimated transformed hidden spectral densities 



variation on (0, oo] 2 , then consider all the (g) triplets of components of Z and their regular variation 
on (0,oo] 3 , and so on. This alternative method requires considering regular variation on 2 d — 1 
cones, whereas our HRV formulation requires considering regular variation on at most d cones. 
The alternative method is difficult to apply in high dimensions. Obviously, there is considerable 
flexibility in choosing a nested sequence of cones and informed choice by a practicioner will be 
governed by the application. 

Another potential defect of our formulation of HRV is that it is designed to deal with the kind of 
degeneracy which arises when the limit measures are concentrated on the axes, planes etc. But the 
limit measures might exhibit different kind of degeneracies. For example, consider the degeneracy 
in the case of complete asymptotic dependence, where the limit measure is concentrated on the ray 
{x € E : x l = x 2 = ■ ■ ■ = x d }. One might think of removing the ray and considering hidden regular 
variation on the cone E\{x G E : x 1 = x 2 = ■ ■ ■ = x d }. Our HRV discussion does not address this 
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Figure 6. Plots of estimates of P[Size > 2 x 10 7 ,Rate > 10 5 ] for different values 
of k using our estimator and H-R(Heffernan-Resnick [15]) estimator 
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Figure 7. Plots of estimates of P[Size > 2 x 10 s , Rate > 10 6 ] for different values 
of k using our estimator and H-R( Heffernan-Resnick [15]) estimator 

issue and we are currently actively thinking about this as well as methods of unifying theories of 
HRV and the conditional extreme value model [17, 16, 7, 6, 14, 13]. 

Other variants of our formulation are possible. The hidden variation on a subcone could be of 
extreme value type other than regular variation and even if we focus only on the hidden variation 
being regular variation, one could envisage different scaling functions for the hidden variation. 
These topics are also being actively considered. 

In estimating the limit measure of hidden regular variation on E^, 2 < / < d, we have 
suggested a method that exploits the semi-parametric structure of . Also, we have constructed 
a consistent estimator of z/™ which relies completely on non-parametric methods, as given in (3.10). 
Our numerical experiments in Section 7 clearly suggest the method exploiting the semi-parametric 
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structure is superior, presumably because it uses more available information about the limit measure 
v®. However, we have no precise, provable comparison. 

An important statistical issue is we have only developed parameter estimators which are con- 
sistent. We have not yet developed theory which allows one to report on confidence intervals for 
parameter estimates or risk probability estimates. 

For characterizations of hidden regular variation, it is important to identify when ({x £ E^ : 
||x|| > 1}) is finite. We found a moment condition to check this, but it requires knowledge of the 
hidden angular measure S"'. A similar problem appeared in checking the finiteness of the right 
side of (6.12). It would be useful to have a statistical test for finiteness. 
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Appendix A 

Proof of Proposition 3.1. The idea of the proof is similar to Proposition 2 of [26]. Define E^^ = 
E") \ Di<i 1< i 2< ...i l <d [x n = oo, x 12 = oo, • • • , x H = oo] and E 2 = (0, oo) x 5^ l \ Define a continuous 
bijection Q^ l > : Ea^ — > E 2 as in (3.4). We first show the equivalence of the vague convergence of 
measures restricted to Ea^ and E 2 , and then extend the convergence to the corresponding whole 
spaces using the scaling property. 

Step 1: First, we prove the direct part. So, we suppose that (2.10) holds with j/W(^(0) = 1. 
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Hence, the convergence also holds with the measures being restricted to En^, i.e. 
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Now, we proceed to show that for each compact set K2 in E2, (Q(0) 1 {K2) is a compact set of 
E;\cxd- Note that the compact sets in are those closed sets K for which every x G K satisfies 
the property that r < x^ 1 ' < s for some < r < s [28, page 170]. Take a compact set if 2 in E2. 
We claim that K2 must be contained in a set K2 of the form K2 = [r,s] x <5^(0. Now, from the 
description of the compact sets of Ea^, (Q(0) (-K2) = {x G E(0 : r < < s} is compact in 
EjVjuj. Also, since Q(0 is continuous, (Q(0) ^A^) is closed. Therefore, (Q(0) ^A^) is a closed 

subset of the compact set (Q(0) (K2) and hence is compact in Ea^. So, using Proposition 5.5 
(b) of [28] we get 

7(0 

4 i> a(l) x 5«(TIE 2 ) inM + (E 2 ). 



iP 



6(0 (i)' 7(0 



g -nE 2 



Now, we want to extend the convergence over the whole space (0, 00] x 8$P\ Choose any relatively 
compact subset A of fitifi' such that S^(5A) = and choose s > r > 0. Then, 
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The first equality in the above set of relations follows from (2.10). Hence, from (A.2) and (A.3), 
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Hence, from (A.l) and (A. 4), we conclude the direct part of the proof: 
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Step 2: To see the converse, again we prove first the vague convergence of the restricted measures 
in M+fE/Xgo) and then extend it to convergence of measures in M + (E(0). We assume that (3.1) 
holds. Restriction on E2 gives 
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First we note that the compact sets of E 2 are those closed sets K for which every (w, v) G K 
satisfies the property that r < w < s for some < r < s. Take a compact set i^i of Ea^. Observe, 
from the description of the compact sets of E^^ as given before, that K\ must be contained in 
a set K\ of the form K\ = {x € E(0 : G [r, s]}. From the description of compact sets of E 2 , 
Q^(Ki) = [r,s] x SH® is compact in E 2 . Since (Q^) is also continuous, the set Q^(Ki) is 
closed, and hence, being a closed subset of a compact set Q( l \Ki), is compact. Therefore, using 
the continuous map 

(Q(0) : E 2 -> E^qo and Proposition 5.5 (b) of [28], we get 
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Now, we want to extend this convergence over the whole space E(0. Choose a relatively compact 
set A of E® such that z/(0(<L4.) = 0. Note that, from the description of relatively compact sets in 
as given in Section 2.2.2, A C {x G E(0 : x(0 > r} for some r > 0. Also, from the earlier 
description of compact sets of Ejw,, it follows that for all s > r, A n {x G E^' : x(0 < s} is a 
relatively compact set of E^^ , and 

i/(0(,5(A n {x G EW : x ( < s})) < i/W(<5A) + i/ {0 ({x G E^ : 
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The second equality in the above set of relations follows from (3.1). Hence, from (A. 6) and (A. 7), 
we get 
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Therefore, from (A. 5) and (A. 8), we conclude 
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which completes the converse part of the proof. 
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Proof of Proposition 3.7. The idea of this proof is similar to Theorem 6.1 of [28, page 173]. Define, 
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the continuous bijection Q(0 ; Ea^ — > E 2 as in (3.4). As in Proposition 3.1, here also the idea of 
the proof is to first show the equivalence of the weak convergence of random measures restricted to 
M + (E;y 0O ) and M+(E2), and then extend the convergence to the corresponding whole spaces using 
the scaling property. 

Step 1: First, we prove that (3.11) implies (3.12). The convergence in (3.11) implies 
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on M + (E 2 ). To extend the convergence to the space (0, oo] x <5^(0 , we use the convergence of laplace 
functionals and use Theorem 5.2 of [28, page 137]. Take / € C£((0,oo] x 8&®) t where C£(F) is 
the set of all continuous functions with compact support from F to IR + . To relate this function to 
one defined in Ci((0, oo) x 5^(0), for all 5, M > 0, we define a truncation function 

1 if < t < M 

if t > M + 5, 

linear interpolation if M < t < M + 5. 
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Since, /^m £ C^((0;°°) x tfN^), by (B.l), we get lim n ,_ >00 B = 0. Now, we proceed to show that 
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exp 



i=l 



exp 



lim sup E 

n— >oo 



1 n 

- k J2f(m?/m%, ((1/rO/mf , 1 < j < dj) 
i n 

l £ Am (mW/m®, ((l/r/)/m?\ 1 < j < <Z 
=i 

litf { m< t ] / m tv ((Vr/Vmj , 1 <i < d 

i=l 
1 - 

T E(Am - /) (mP/m®, ((l/rf')/mf , 1 < j < d 



- exp 
1 — exp 



< lim sup E 

n— >oo 



which, using the facts that ||/|| = sup (r . 9)g(0oo]x5H (o f(r,6) < oo, \\fs,M ~ f\\ < 
and (fsM — f)( x i 6*) = for x < M, is bounded by 



!><5,M 



< 



lim sup E 

n— ¥oo 



E 



1 — exp 
1 — exp 



1 - 

tII/IIE^) M g)([m,oo]) 



1=1 



i ' (A:) 



1 " 



fc^ e ((lAD/m«,l<J<d 



({x G : x W E [M, oo]}) 



which, by (3.11), converges as n — > oo, to 



1 — exp 
= 1 — exp 
= 1 — exp 



II/H- i/W({x€EW G [M,oo]}) 

(0 



0. 



as M — > oo. The argument for UniM-too C = is similar and is omitted. Hence, by Theorem 5.2 of 
[28, page 137], we obtain (3.12). 

Step 2: To see the other part, i.e. (3.12) implies (3.11), we use a similar method. The conver- 
gence in (3.12) implies 

1 n 
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on M+(E2). It is easy to see that (Q®) 1 is a continuous bijection. Also, as shown in the proof 
of Proposition 3.1, for any compact set K\ C En^, Q^>{K\) is compact in E2. Therefore, using 
Proposition 5.5(b) of [28] we get 

1 n 

(B.2) *>«)((.) n V ) ^^(^^.K^,) 1 ^/^^, =► "«((■) niVO 

on M + (E;\ QO ). We use the same truncation function to relate functions on Cj£(E^) to ones 

in C7+(E, Xoo ). Choose / G C+(E«). Note that / 5 , M (x) := /(x)^ M (x«) G C+(E ZXoo ) for all 
5, M > 0. 



exp 



<\E 
+ \E 



■fcE/(( 1 /»f)/«»f2), i<j<d 



i=l 



cxp 



exp 



exp 



1 

fc Z) / ((1/ff )/m{ , 1 < J < d 

i=l 
1 n 

EAm ((lM)/r4\l<j<d 



exp 



-zE/m/((iadm!°, i<i<rf 



i=l 



k 



i=l 



cxp 



' (0 (/<w) 



+ |exp[-i/W(/j lM ; 
= A + B + C. 



exp 



Since, fs : M G C^-(^\oo)) by (B.2) , we get lim ri _ ! . 0O B = 0. Now, we will show that Hium-hx bin sup n _ s . oc A 
0. 



lim sup I E 



exp 



1 n 1 1 n 

^/((V^)/™?, l<J<rf) -exp --Y,fs,M^/4)/mf\ l<j<d 
i=l J L i=l 



lim sup £7 

71— ^OO 



exp 



< lim sup £7 

n— >oo 



- exp 
1 — exp 



1 1 h 

i=l 
1 n 

i=l 

1 n 



i=l 



which, using the facts ||/|| = sup xgE( o /(x) < 00, \\fs,M ~ f\\ < 
(fs,M ~ /)(*) = for {x G E w : x® < M}, is bounded by 



^><5,M - 111 < 



and 



1 — exp 



lim sup E 

n— >oo 



n 

rll/llE' 



i=l 



e/, (l/^)/m«,l<i<dJ 



({xGE W G [M,oo]}) 



1 — exp 



1 " 

^E e m f /m (o([M,oo]) 

i=l 
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which, by (3.12), converges as n — > oo, to 
l-exp[-||/||-^ {;) ([M,oo])] 



1 — exp 



M"« (i) 



0. 



as M — > oo. The argument for limM->-oo C = is similar and is omitted. Hence, we obtain (3.11) 
and this completes the proof. □ 
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